Skip to content

Introduce parser_errors spec test suite#137

Merged
tobi merged 2 commits into
mainfrom
liquid-spec-errors
May 20, 2026
Merged

Introduce parser_errors spec test suite#137
tobi merged 2 commits into
mainfrom
liquid-spec-errors

Conversation

@karreiro
Copy link
Copy Markdown
Contributor

@karreiro karreiro commented May 15, 2026

Overview

liquid-spec doesn't have many cases covering syntax errors. This PR adds ~2K specs covering nearly every SyntaxError branch across all tags — all additive YAML, no Ruby changes.

Each spec follows this shape:

- name: "assign - missing variable name"
  template: "{% assign = 'hello' %}"
  errors:
    parse_error: "[Liquid syntax error] Syntax Error in 'assign' - Valid syntax: assign [var] = [source]"

How adapters opt in

The suite is opt-in, so existing adapters see nothing new. An adapter that supports strict2 parsing enables the full suite by adding it to their suite list; the existing feature-gating system handles the rest.

Coverage

The specs aim to cover every raise SyntaxError call site in the parser including unguarded parser.consume(type) calls.

@karreiro karreiro force-pushed the liquid-spec-errors branch from 54890e5 to 165af5c Compare May 15, 2026 12:27
The error matcher (adapter_runner.rb#matches_patterns?) does case-insensitive
substring/regex matching against both the full error text and the extracted core
message. Patterns only need to capture the semantic essence of the error.

The recorded patterns embedded the input template verbatim, e.g.

  'Liquid syntax error: Expected end_of_string but found close_round in "x = x )"'

That overfits to liquid-ruby's exact wording AND to the exact template text,
which means any implementation with slightly different quoting/formatting fails.

Trim 1901 patterns down to their semantic core:

  'Expected end_of_string but found close_round'

This matches the existing convention in specs/liquid_ruby/strict_rejects_lax_syntax.yml
(short substrings like 'but found pipe', 'Expected close_square').

Result: 80 unique patterns, max length 95 chars (was 1901 unique, often >100).
Verified against liquid_ruby adapter: 1791/1901 specs match correctly; the 110
remaining failures are specs whose enforcement requires strict2_parsing.
@tobi tobi merged commit f49e517 into main May 20, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants